a) Field of the Invention
The present invention relates to techniques of processing image data output from an image sensor, and more particularly to an image processing apparatus and method which can compensate for defects of pixels of an image sensor through digital data processing.
b) Description of the Related Art
FIG. 5 shows the structure of a general image processing apparatus. An image processing apparatus may be a digital camera. Image data output from an image sensor (CCD) 10 is sequentially written in a DRAM 11, for example, one line after another. Pixel data of all lines, i.e., image data of one frame, is temporarily written in DRAM 11.
The image data of one frame once written in DRAM 11 is read under the control of a host CPU 12 to write it in another storage medium, to output it to a display device, or to further process it and output the result.
An image sensor 10 manufactured in a semiconductor factory has defective pixels with some percentage. Even if there are some defective pixels among several hundred thousand pixels or several million pixels of each product, such defects are permitted as commercial products. Since pixel defects lower the image quality, it is preferable to correct the defects by some means. For example, data of defective pixels are corrected by digital processing techniques or the like.
The positions of defective pixels of an image sensor can be known at the stage of manufacture inspection. The position information of defective pixels is supplied from a manufacturer of image sensors, and stored in an external memory 13. The position information of defective pixels is supplied from the external memory 13 to the host CPU 12 which executes a process of correcting data of defective pixels of the image sensor in the following manner.
Image data of one frame supplied from the image sensor 10 is once written in DRAM 11 together with image data of defective pixels. Next, the position information of defective pixels of the first one line is fetched from the external memory 13 to correct the data of the defective pixels designated by the defective pixel position information. This defective data correction process is performed for all lines containing defective pixels.
Referring to FIGS. 4(A) and 4(B), a process of correcting defective pixel data will be detailed. FIG. 4(A) is a schematic diagram showing a layout of some pixels in one line of a monochrome image sensor. A pixel 21 indicated by a cross mark is a defective pixel. In accordance with the defective pixel position information, the host CPU 12 reads the data of normal pixels 20 and 22 one pixel before and after the defective pixel 21, from DRAM 11. An average of the data of the pixels 20 and 22 is calculated. The data at an address of the defective pixel 21 in DRAM 11 is replaced by the calculated average data. This data correction process is executed for all lines containing defective pixels, and the data of defective pixels in DRAM 11 is corrected.
FIG. 4B is a schematic diagram showing a layout of some pixels in two lines of a color image sensor. This pixel layout is called a Bayer matrix. In the Bayer matrix, a row alternately disposing an R (red) pixel and a G (green) pixel and a row alternately disposing a B (blue) pixel and a G (green) pixel are alternately disposed in the column direction.
It is assumed that an R pixel 31 indicated by a cross mark is a defective pixel in a subject line n. In accordance with the defective pixel position information, the host CPU 12 reads the data of normal R pixels (same color as the defective pixel) forward and backward nearest to the defective pixel 31, from DRAM 11. An average of the data of the two normal pixels 30 and 32 is calculated. The data in DRAM 11 at an address of the defective R pixel 31 is replaced by the calculated average data. Defective data for the B and G pixels is corrected by the similar process. This data correction process is executed for all lines containing defective pixels, and the data of defective pixels in DRAM 11 is corrected.
Image data of one frame subjected to the defect data correction is read from DRAM 11 to follow a next process of reading the image data in another storage, to output it to a display device, or to further process it and output the result.
The image processing apparatus described above executes the defect pixel data correction process after image data of one frame from the image sensor is once written in DRAM 11. The operation of reading data of normal pixels near the defective pixel from DRAM 11, executing the defective pixel data correction process, and overwriting image data in DRAM 11 takes a process time of about 10 clocks.
Since the defective pixel data correction process is repeated as many as the number of defective pixels, it takes a fairly long to calculate the correction data for all defective pixels and write it in DRAM. This results in a long time taken to correct image data of one frame picked up with an image sensor with defective pixels and output corrected image data. There is therefore a long wait time for a picked up image to be displayed. Since the time necessary for the defective pixel data correction is long, this correction is not applied to a monitor moving image having a frame rate of 30 frames per sec.